Incorporating Non-verbal Modalities in Spoken Language Understanding for Multimodal Conversational Systems
نویسندگان
چکیده
INCORPORATING NON-VERBAL MODALITIES IN SPOKEN LANGUAGE UNDERSTANDING FOR MULTIMODAL CONVERSATIONAL SYSTEMS
منابع مشابه
Langage de conversation multimodal pour agent conversationnel animé
This article falls within the realm of dialogue between a human and an Embodied Conversational Agent (ECA). We claim that a specific agent conversational language is needed for such interactions, based on the essential role of emotion in human communication. In order to define this language, we propose a library of Multimodal Conversation Acts is proposed, based in particular on speech acts and...
متن کاملHuman Language Technology Conference of the North American Chapter of the Association of Computational Linguistics Proceedings of the Doctoral Consortium
Structural information in language is important for obtaining a better understanding of a human communication (e.g., sentence segmentation, speaker turns, and topic segmentation). Human communication involves a variety of multimodal behaviors that signal both propositional content and structure, e.g., gesture, gaze, and body posture. These non-verbal signals have tight temporal and semantic lin...
متن کاملUsability Evaluation of Multimodal and Domain-Oriented Spoken Language Dialogue Systems
Considerable work has been done regarding usability evaluation of task-oriented unimodal spoken language dialogue systems (SLDSs). However, there are still important gaps in our knowledge even in this area. If we move to multimodal task-oriented SLDSs, there are more challenges ahead primarily due to the combination of different modalities. For non-task-oriented conversational SLDSs, a major ch...
متن کاملA Multimodal Corpus of Rapid Dialogue Games
This paper presents a multimodal corpus of spoken human-human dialogues collected as participants played a series of Rapid Dialogue Games (RDGs). The corpus consists of a collection of about 11 hours of spoken audio, video, and Microsoft Kinect data taken from 384 game interactions (dialogues). The games used for collecting the corpus required participants to give verbal descriptions of linguis...
متن کاملIncorporating Gesture and Gaze into Multimodal Models of Human-to-Human Communication
In human communication, utterances are expressed with some structural events, e.g., sentence, speech repairs, control of floor, and etc. These structural events bring important information and quite helpful for a better understanding of human communication. Meanwhile, the human communication is also full of multimodal behaviors, e.g., gesture, gaze, and etc. As non-verbal signals, gesture and g...
متن کامل